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Background: With an ageing population and increasing demands on health and social care 
services, there is growing importance attached to the management of long-term conditions, 
including maximizing the cost-effectiveness of treatments. In line with this, there is increasing 
emphasis on the need to keep people both active and participating in daily life. Consequently, 
it is essential that well developed and validated instruments that can meaningfully assess lev- 
els of participation and activity are widely available. Current measures, however, are largely 
focused on disability and rehabilitation, and there is no measure of activity or participation for 
generic use that fully meets the standards set by regulatory bodies such as the US Food and 
Drug Administration. Here we detail a protocol for the development and validation of a new 
patient-reported outcome measure (PROM) for assessment of participation and activity in 
people experiencing a variety of health conditions, ie, the Oxford Participation and Activities 
Questionnaire (Ox-PAQ). The stages incorporated in its development are entirely in line with 
current regulations and represent best practice in the development of PROMs. 
Methods: Development of the Ox-PAQ is theoretically grounded in the World Health 
Organization International Classification of Functioning, Disability, and Health. The project 
incorporates a new strategy of engaging with stakeholders from the outset in an attempt to identify 
those characteristics of PROMs considered most important to a range of potential users. Items 
will be generated through interviews with patients from a range of conditions. Pretesting of the 
instrument will be via cognitive interviews and focus groups. A postal survey will be conducted, 
with data subject to factor and Rasch analysis in order to identify appropriate dimensions and 
redundant items. Reliability will be assessed by Cronbach's alpha and item-total correlations. 
A second, large-scale postal survey will follow, with the Ox-PAQ being administered in con- 
junction with generic measures of health status to further test the validity of the measure. The 
Ox-PAQ will again be administered at 2 weeks to assess test-retest reliability and at 3 months 
to assess responsiveness. 

Conclusion: The development of the Ox-PAQ is a timely one. With increasing emphasis being 
placed on the importance of keeping people active and participating in daily life, the instrument 
has the potential for significant uptake. Its primary use is intended to be in clinical trials and for 
evaluation of interventions targeted at maintaining activity and participation. 
Keywords: activity, participation, patient-reported outcome measure, questionnaire 

Introduction 

With an ageing population and increasing demands on health and social care services, 
growing importance is attached to the management of long-term conditions, including 
maximizing the cost-effectiveness of treatments. In line with this, there is increasing 
emphasis on the need to keep people both active and participating in daily life. M 
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Consequently, it is essential that well developed and validated 
instruments that can meaningfully assess levels of participa- 
tion and activity are widely available. 

The current literature, however, demonstrates little 
consensus on how we define "participation" and "activity". 
A number of papers have been published with the aim of 
clarifying or reaching agreement on a clear definition for 
both terms, 5 " 7 but with limited success. Some have called for 
greater clarity between the two concepts, 8 whilst others have 
concluded that participation is a complex, elusive, and sub- 
jective concept, influenced by one's individual perspective. 9 
This serves to highlight the importance of investigating the 
meaning of concepts such as participation and activity with 
those in whom we are trying to measure it. As previous quali- 
tative research demonstrates, when this is undertaken, the 
complex nature of what, for example, participation actually 
means to different individuals becomes evident. 10 

One widely used definition of the concepts of participation 
and activity is to be found in the World Health Organization 
International Classification of Functioning, Disability and 
Health (ICF), 11 which at least provides a framework within 
which to construct scales aimed at measuring the two 
constructs. The ICF defines participation as "involvement 
in life situations" and activity as "the execution of a task or 
action by an individual". Although initially described as two 
separate concepts, the final version of the ICF merges activity 
and participation into a single taxonomy and identifies nine 
domains which are presented in Table 1 . 

There is still significant debate over the ICF model and 
particularly in relation to whether activity and participation 
is a single entity, or whether a clear distinction should be 
made between the two. 1213 Despite this continuing debate, 
a number of patient-reported measures of activity and par- 
ticipation have now been constructed, to varying degrees, 
based on the model. 14 Recently published reviews and clinical 
commentaries, however, highlight significant limitations in a 
number of these measures. 15 " 20 Current measures are largely 

Table I Domains of the ICF classification of participation and 
activity 

1 . Learning and applying knowledge 

2. General tasks and demands 

3. Communication 

4. Mobility 

5. Self-care 

6. Domestic life 

7. Interpersonal interactions and relationships 

8. Major life areas 

9. Community, social, and civic life 

Abbreviation: ICF, International Classification of Functioning, Disability and Health. 



focused on disability and rehabilitation, with many falling 
short of the full range of psychometric properties required 
for patient-reported outcome measures (PROMs) under 
guidance such as that provided by the US Food and Drug 
Administration (FDA). 21 The development of a number of 
instruments is poorly reported and some have been devel- 
oped with small sample sizes, which calls into question the 
legitimacy of the psychometric analyses undertaken. Many 
have failed to involve patients in generating items, some- 
thing that is considered fundamental in developing relevant 
PROMs that accurately reflect the concerns of those who 
are being assessed. 21 

Such a picture provides a sound rationale for develop- 
ing a new measure, the Oxford Participation and Activities 
Questionnaire (Ox-PAQ). The Ox-PAQ initiative aims to 
address a number of the issues highlighted above by develop- 
ing a measure theoretically grounded in the ICF for generic 
use and with patients at the heart of the item-generation 
process. Additionally, the views of stakeholders will be gained 
from the outset in an attempt to identify those characteristics 
of PROMs considered most important to a range of potential 
users. The measure will subsequently be assessed for the full 
range of psychometric properties, and sample sizes will allow 
for detailed and legitimate psychometric analyses. The devel- 
opment process will be entirely in line with FDA guidelines 
which are now considered best practice, regardless of whether 
an instrument is being incorporated into, or developed for, 
trials supporting pharmaceutical label claims. 22 

Materials and methods 

Ethical approval for the study was granted by the Medical 
Sciences Inter-Divisional Research Ethics Committee of the 
University of Oxford (reference MSD-IDREC-C1-2013- 
064). The study will follow a nine-point development pro- 
cess, as outlined in Table 2. The methodology (participants, 
materials, procedure, and statistical analysis) as to how this 
development process will be achieved is detailed below. 

Table 2 Developmental stages of the Ox-PAQ initiative 



Stage Activity 



1 


Stakeholder interviews 


2 


Patient interviews 


3 


Item generation and pretesting 


4 


Pi lot- test survey 


5 


Psychometric analysis of pilot test survey 


6 


Large-scale field test 


7 


Psychometric analysis of field test 


8 


Test-retest assessment 


9 


Responsiveness assessment 



Abbreviation: Ox-PAQ, Oxford Participation and Activities Questionnaire. 
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Participants 

Development of the Ox-PAQ will require three separate 
groups of participants. The source of recruitment and nature 
of participants will depend on the particular stage of the 
project as outlined below. 

Stakeholder interviews 

A broad range of participants has been drawn from a number 
of areas where patient-reported outcomes are of relevance 
(ie, medicine, academia, regulation). 

Patient interviews 

A sample of patients (approximately 3CM10) drawn from a 
wide range of conditions, including arthritis, cancer, chronic 
back pain, diabetes, motor neurone disease, multiple scle- 
rosis, Parkinson's disease, and spinal cord injury, will be 
recruited via relevant support charities. The patient groups 
included represent a spectrum of conditions that together 
affect a number of key bodily systems, as well as having 
different symptoms, trajectories, and prognoses. 

Survey participants 

A large separate sample of patients with the conditions 
outlined above will again be recruited via relevant support 
charities. 

Inclusion/exclusion criteria 

Participation in the stakeholder interviews requires partici- 
pants to have some knowledge and previous experience with 
PROMs, and be actively engaged in a field where PROMs 
are of relevance. Inclusion in the patient interview phase 
will require participants to have a confirmed diagnosis of 
the condition being investigated. Survey participants will be 
excluded if they are unable to complete the questionnaires 
independently and if they do not have English as a primary 
language. All participants must be aged 18 years or over. 

Materials 

Two instruments will be administered alongside the Ox-PAQ 
for the purpose of evaluating its validity. 

The SF-36 

The SF-3 6 23 - 24 is a 36-item instrument containing eight 
domains (physical functioning, role physical, role emotional, 
social functioning, mental health, energy/vitality, pain, and 
general health perception) that has been extensively used 
and shown to have excellent psychometric properties. 25,26 
Raw scores for each domain are transformed to have a range 



from 0 to 1 00, with higher scores indicative of superior health 
status. Two summary scores can also be calculated from the 
results of the eight domains, ie, the Physical Component 
Summary and Mental Component Summary. 27,28 

The EQ-5D 

The EQ-5D 29 is a generic, five-item questionnaire that also 
incorporates a visual analog scale (ranging from 0 [worse 
health] to 100 [best health]) to indicate general health status. 
A unique health state is derived by combining responses from 
each of the five questions. Each score is intended to reflect 
societal valuations of the health state in terms of "utility" or 
valuations of health-related quality of life. Any one of the 
possible 243 health states that can be derived is converted 
to a summary score, with higher scores indicating superior 
quality of life. The instrument has acceptable reliability for 
both the utility (/-0.73) and visual analog scale (r=0.70) 
components. 30 

Procedure 

The procedure for the nine developmental stages outlined in 
Table 2 will be conducted as follows. 

Semistructured interviews have been conducted across 
Europe with 18 professionals from a range of different 
backgrounds and areas of expertise. These have included 
regulation, commissioning, reimbursement, clinical prac- 
tice, nursing, health economics, national government, aca- 
demia, clinical trials, patient support, and health technology 
assessment. The purpose of these interviews was two-fold: 
firstly, to discuss issues surrounding current PROMs and, 
secondly, to consider the broad topic areas a generic mea- 
sure of participation and activity might include that will be 
meaningful and considered important. No previous measure 
has been designed from the outset with the views of such a 
variety of potential users at its core. Furthermore, the inter- 
views discussed the practicalities of the instrument, such as 
preferred methods of scoring and rules for interpretation. The 
interviews will be transcribed, subjected to content analysis, 
and used to inform later stages of the project. 

Exploratory, open-ended, face-to-face, indepth interviews 
will be undertaken with a sample of patients across a wide 
range of conditions. The interviews will aim to identify 
salient dimensions of illness impact that have adversely 
affected participation and activities. It is anticipated that 
approximately 30-40 patients will be interviewed in 
depth until, as recommended, "no new themes emerge". 31 
Interviews will be recorded, transcribed, and subjected to 
content analysis. Whilst there will be an interview guide, 
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participants will be free to range across this and address any 
other relevant topics. 

The interviews conducted in Stage 2 will be transcribed 
and transcripts will be scrutinized independently by the 
research team for issues relating to illness impact. Issues 
arising will be recast as questionnaire items to be answered 
on a five -point Likert scale. A Flesch readability score 32 will 
be calculated for each item with a target of approximately 
grade 6. A meeting of health care researchers and relevant 
interested stakeholders will be convened to assess the list 
of items for completeness, ambiguity, and repetition. The 
resulting candidate questionnaire will then be discussed with 
a small group of people representing a range of conditions 
in a focus group setting for their comments and to assess its 
face validity. Additionally, the questionnaire will be pretested 
with a variety of patients in cognitive interviews using the 
format outlined by Willis. 33 Participants will be asked to 
complete the questionnaire and explain the reasons for their 
answers to each question, and comment upon any difficulties 
or ambiguities they experience. 

The long-form questionnaire drawn up in Stage 3 will 
be mailed to a sample of 600 people with a variety of long- 
term conditions. This sample size is based on the assumption 
that the instrument will contain approximately 50 items. 
Estimates suggest that a minimum of five times as many 
respondents than items are required for psychometric tests to 
be meaningful. 34 Assuming a typical response rate of between 
60% and 70%, 35 this will lead to a sample of approximately 
360^420 questionnaires, which will permit rigorous testing 
of the instrument. 

Analysis of data from Stage 4 will allow for identification 
of a shorter form questionnaire using established statisti- 
cal procedures as detailed below in the Statistical analysis 
section. Items with high face validity, good correlation with 
the scale total to which they contribute, and sound reliability 
will constitute a dimension of the resulting instrument. 

The short-form measure generated in Stage 5, together 
with the SF-3 6 23 - 24 and EQ-5D, 29 will be mailed to 
1,000 patients with a wide variety of conditions. Assuming 
a 60%-70% response rate (as discussed previously), this will 
yield a final sample size of approximately 600-700 returned 
questionnaires. 

Further statistical analyses will be undertaken on data 
from Stage 6 to confirm the results from the initial survey in 
Stage 5. Minor amendments may be made to the question- 
naire at this time. 

Previous respondents will be asked to complete the 
questionnaire once more. If they agree to do so, they will 



be sent the Ox-PAQ one week after receipt of the measure 
from Stage 6 in order to assess test-retest reliability. Previous 
respondents will be asked to complete the questionnaire for 
the last time after 3 months has elapsed in order to assess the 
instrument's responsiveness. 

Statistical analysis 

Data will be checked for presence of outliers and normality of 
distribution prior to statistical analysis. Distribution of the data 
will be analyzed using the Shapiro-Wilk test of normality. 36 
Subscale and summary scores for the SF-36 and EQ-5D will 
be calculated according to respective scoring algorithms. 
Items of the Ox-PAQ will be assessed for floor and ceiling 
effects under the criteria outlined by Terwee et al. 37 Factor 
analytic techniques will subsequently be performed to assess 
construct validity and identify dimensions relating to specific 
aspects of participation and activity. Item-total correlations, 
corrected for overlap, will be calculated between items and 
the total score to which they contribute. Rasch analysis 38-40 
will also be conducted to determine which items conform 
to a hierarchical unidimensional structure. Further tests of 
validity (concurrent and discriminant) will be measured 
by calculating Pearson's product-moment 41 or Spearman 
rank correlation 42 coefficients (as appropriate depending on 
data distribution) between the Ox-PAQ, SF-36, and EQ5D. 
Reliability will be assessed via tests of internal consistency 
using Cronbach's alpha coefficient 43 and test-retest reliability 
using intraclass correlations. 44 Responsiveness to change will 
be determined through calculation of standardized effect 
sizes. 45 Data will be coded in order to maintain confidenti- 
ality and analysed using Statistical Package for the Social 
Sciences version 20 software (IBM, Armonk, NY, USA). 
Rasch analysis will be conducted using RUMM2030. 

Discussion 

This paper has described the initial stages of developing 
and validating the Ox-PAQ, a new measure of participation 
and activity. The rationale for developing this instrument 
is relatively straightforward. Current measures are largely 
focused on disability and rehabilitation, and there is no 
measure of activity and participation for generic use that 
fully meets the standards set by regulatory bodies such 
as the FDA. The Ox-PAQ initiative aims to ameliorate 
this. The measure will be broadly focused and for use in a 
range of conditions. The developmental stages discussed 
follow current guidelines and represent best practice in the 
development of outcome measures. Additionally, the proj- 
ect incorporates a new strategy of involving and engaging 
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with potential users from the outset of the development 
process. 

The development of the Ox-PAQ is a timely one. With 
increasing emphasis being placed on the importance of 
keeping people active and participating in daily life, the 
instrument has the potential for significant uptake. Its 
primary use is intended to be in clinical trials and related 
forms of evaluation of interventions targeted at maintaining 
activity and participation. Given the increasing importance 
of electronic data capture, the development of an electronic 
version of the Ox-PAQ, for use on platforms such as comput- 
ers, tablets, and smartphones, is something that may follow 
its initial development and validation. The progress of the 
development and validation of the Ox-PAQ can be followed 
at the University of Oxford Health Services Research Unit 
website ( http://www.dph.ox.ac.uk/research/hsru/OxPAO ). 
and results will be disseminated via international confer- 
ences and journals. 
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